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Abstract 

We show optimality, in a well-defined sense, using cumulative sum (CUSUM) charts for 
detecting changes in distributions. We consider a setting with multiple changes between two 
known distributions. This result advocates the use of non-restarting CUSUM charts with 
an upper boundary. Typically, after signalling, a CUSUM chart is restarted by setting it to 
some value below the threshold. A non-restarting CUSUM chart is not reset after signalling; 
thus is able to signal continuously. Imposing an upper boundary prevents the CUSUM chart 
rising too high, which facilitates detection in our setting. We discuss, via simulations, how 
the choice of the upper boundary changes the signals made by the non-restarting CUSUM 
charts. 
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1 Introduction 



In statistical process control, one of the most commonly used control charts is the cumulative 
sum (CUSUM) chart developed by Page (1954). As a simple example of a CUSUM chart 
consider the following. Assume we sequentially observe the independent random variables Xt 
(t € N = {1, 2, . . . }). Suppose the observations X±, . . . , X„_i each have distribution N(0, 1) and 
X n , X n+ i, . . . have N(A, 1) for some known A > 0. The aim is then to find the unknown change 
point n £ N. The classic CUSUM chart is 

S t = max{S t -i + X t - A/2, 0}, S = 0. 

The chart signals a change at time inf{t > 0; St > a} for some threshold a > 0. At this time 
the chart suspects a change in distribution. When the CUSUM chart crosses the threshold a, 
it is restarted in some fashion. The chart is typically reset at 0, with some practitioners opting 



for the headstart feature (Lucas and Crosier, 1982), where the chart is reset at a different value 



such as a/2. CUSUM charts were initially designed for use in industry (Page 1954), where 
restarting coincides with a machine being repaired and reset. 

We are concerned with situations where the observations can switch multiple times between 
two known distributions and where resetting is not possible. For example, medical settings where 
"machines" such as hospitals cannot be reset when a deterioration of performance is suspected. 
Thus we shall use the CUSUM charts of the form 

R t = ft(R t -i) where f t (x) = min{max(x + log£(X t ), 0), h} (1) 

for t £ N, where £{x) is the Radon-Nikodym derivative of Fi with respect to Fq and h > 



is a constant specifying an upper boundary. As explained in Gandy and Lau (2012), a single 



non-restarting CUSUM chart ([!]) with Rq = is appropriate in settings where restarting is 
not possible. Moreover, it is shown that the imposition of an upper boundary facilitates the 
detection when switching between an in-control state and out-of-control state many times. The 
CUSUM charts defined by gj constitutes a family of charts, each member represented by a 
specific choice of starting value Rq £ [0, h]. We distinguish two particular CUSUM charts which 
we shall use throughout this paper. 

Definition 1 For a specified upper boundary h > 0, denote the CUSUM chart of the form ([!]) 
with Rq = as R\ and with Rq = h as R^ . 

These are the "extreme" charts in ([!]) i.e. for any Rq £ [0, h] we have R\ < Rt < RY for all 

ten. 

In this paper, it is shown that using CUSUM charts of the form ([I]) are optimal in a well- 
defined sense. The optimality criteria we consider is similar to that used by Lorden (1971). 



Our optimality is a generalisation of the Moustakides (1986) result, where a single change in 



distribution is considered. Showing optimality of CUSUM charts is a subject of interest in the 



change point detection literature. For instance, Poor (1998) extends the optimality in the sense 



of Lorden ( 1971 ), to include a exponential penalty for delay. Moreover, extensions to continuous 



time processes (Moustakides 2004), use of dependent observations such as Markov chains (Yakir 
1994) and random processes (Moustakides 1998) have also been explored. However, to our 



knowledge, the optimality of CUSUM charts in a setting where observations can switch between 
two known distributions multiple times has not been investigated. 

To make clear our setting and the charts we shall being using, consider the following illus- 
trative example. Let Xt ~ N(— 1/2, 1) when in the in-control state and Xt ~ 2V(l/2, 1) when in 
the out-of-control state at time t. In this example, we use the charts R\ and R^ with h = 16 
with threshold a = 8. These non- restarting CUSUM charts will signal out-of-control at time t if 
R\ > a. Moreover, these charts will continually signal out-of-control whilst R\ remains above 
a. Similarly, for signalling in-control when the upper chart R^ drops below a. 

In Figure [T] it is clear that the lower boundary at and upper boundary at h are "holding 
barriers" that prevent the chart dropping too low or rising too high. Beneath the plot of the 
CUSUM chart in Figure [TJ is the signal indicator. The grey areas represent the charts signalling 
out-of-control and in-control, where it is clear that continuous signals are made. This would not 
be the case when using a traditional CUSUM chart (i.e. a chart of the form ([I]) with h = oo 
starting at 0) that restarts when its threshold is crossed. 

In Section [2~T| we begin by introducing notation appropriate in our setting. We then introduce 
stochastic processes as signal processes, that allows continuous signalling. In Section |2.2| we 



present Lorden ( 1971) criterion of optimality as used in Moustakides (1986). Section 2.3 contains 



the main theoretical result of the paper. Here, it is shown that using non-restarting CUSUM 
charts with an upper boundary are optimal in a well-defined sense. Although the upper boundary 
is introduced in the theoretical result, its value remains unspecified. In Section [3] we explore 
how the choice of the upper boundary affects the signals made and practical consequences. We 
conclude in Section [4] we some remarks concerning future topics of research. 

2 Optimality 



In this section we present our optimality criteria. We present the theoretical result proving that 
using non-restarting CUSUM charts with an upper boundary, are optimal in the sense defined. 



We begin by introducing notation and recalling the Moustakides (1986) result. 
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Figure 1: Two non-restarting CUSUM charts (above) with signal indicator (below). 



2.1 Notation 



Let (£l,F) be a measurable space with random variables (Xt : t € N). Let V be a set of 
probability measures on (O, J 7 ) such that for all P G V, Xt (t E N) are independent and the 
distribution of Xt, C(X t ) E {Fq,F\\ for all t £ N. Both Fq and Fi are assumed to be known 
and mutually absolutely continuous. We will assume that £(X\) has no atoms with respect to 
P G V such that C(X t ) = Fq for all t G N. The cr-algebra generated by {Xi, . . . , X t } is denoted 
by Ft- Henceforth, whenever we use an expectation E or essential supremum, we shall specify 
with respect to which probability measure in V by defining the distributions of the Xt (t EN). 
We shall refer to the case C(X t ) = Fq as X t being in the in-control state and C(X t ) = F\ as X t 
being in the out-of-control state. 

As we are concerned with detecting periods where the observations are in the in-control 
state or the out-of-control state, it is not convenient to work with stopping times as used in 

Thus we shall use a stochastic process as a signal 



Moustakides 


( 


1986 


) and 


Lorden 


( 


1971 



process. 

Definition 2 A signal process is a stochastic process Z = {Zt ■ t E N} such that Zt E {0, 1,0} 
and Zt is Ft -measurable for all t EN. 

For a signal process Z, the events {Zt = j} correspond to signalling in-control (J = 0), signalling 
out-of-control (j = 1) and no signal (j = 0) at time t. The first in-control (j = 0) or out-of- 
control (j = 1) signal by the signal process Z after time n is denoted by Tn(Z) = inf{t > n : 
Z t = j}- 



Lastly, we define CUSUM chart developed by Page (1954) and its analogue, with the 



roles of Fq and F\ swapped as follows. 

Definition 3 For h = oo and ft as in Q let, for all t EN 



St = ft(S t L -i) and S? = -M-SlJ with 5 L 



2.2 iMoustakidesI (119861) Optimality Result 



In this section we present the optimality result of Moustakides ( 1986 ) in terms of signal processes. 



First, we recall optimality criteria used in Lorden (1971). 



Definition 4 Let P £ V he such that C(X t ) 
For a signal process Z and j 6 {0, 1} let 



Fi—j for all t < n and C(Xt) = Fj for all t > n. 



T>i(Z) = esssup£ | t{(Z) - n + 1 + | F n --\ 



B j (Z) = supD£(Z). 



neN 



The term D J (Z) is the longest average delay of signalling the X's are Fj distributed, guaranteed 
regardless of the behaviour of the F±—j distributed X's before the change. 



For a constant 71 > 0, consider the optimization problem posed in Moustakides (1986) 



mm 



T>\Z) 

E{t\{Z) I C(X t ) = Ftft G N} > 71 



(2) 



where Z is a signal process. As proved in |Ritov (1990, Proposition 2) a solution of ^ is the 
signal process Z l where Z\ = 1 if > hi, for t £ N, where ki, > is a constant determined by 



71. We appeal to this alternative proof of Moustakides (1986) to avoid complications with the 



cr-algebras. By swapping the roles of Fo and F\ we obtain an analogous result. More precisely, 
for a constant 70 > 0, consider the optimization problem 



D°(Z) -> min 

E{t°(Z) I C(X t ) = F yt G N} > 70 



(3) 



where Z is a signal process. A solution of ^ is the signal process Z° where Z® = if > ku 
for t € N, where ku > is a constant determined by 70. 

The expectation in ([2]) is average time until signalling out-of-control, when all the observa- 
tions are truly in-control and vice versa for the expectation in ([3]). This is referred to as the 
average in-control and out-of-control run lengths respectively. 

2.3 Main Result 

In this section we present our optimality result. We begin by defining our optimality criteria. 
Definition 5 For a signal process Z and j £ {0, 1} let 

Cl(Z) =max{esssup J B(r^(Z) -n+ 1 | T n -i) : P £V ', £{X t )=F j Vt > n) , 
C>(Z) = supC£(Z). 

neN 

For constants c\ > and C2 > 0, consider the optimization problem 



C l {Z) -»■ min 
C°(Z) -»■ min 

> c 
S 1 ^) > ci 



(4) 
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where Z is a signal process and for j E {0, 1} with P £ V such that C(X t ) = Fj for all i E N 
BHZ) = inf BUZ) where BUZ) = esssup £ W 3 n (Z) - n + 1 I J" n _i} . 

nGN 

The term C 1 (Z) (C°(Z)) is the longest average delay of signalling out-of-control (in-control), 
guaranteed regardless of the distribution of the X f s prior to the permanent change out-of-control 
(in-control) and prior signals made. Unlike D J (Z) which considers the first stopping time t((Z), 
we consider the reset stopping time Tn(Z). Hence it is possible to repeatedly signal in-control 
or out-of-control. Each Bi(Z), n E N, is the in-control (j = 1) or out-of-control (j = 0) average 
run length after the change point n. 

Theorem 1 There exists constants kjj > and ki > such that for any h > max(ku, k^) a 
solution of Q is the signal process Z* defined, for i £ N, by 



Z* 



0, R? <h- kjj 

1, Rf>k L 

0, R\ < k L , R? >h- k v 



The proof of Theorem [T] is in Appendix 1. 

Theorem [l] uses the charts R\ and Rf , which both follow the same evolutionary equation 
ft. However, Rf starts at value whereas R^ starts at the upper boundary h. Starting at these 
two values means that we consider both the cases where the observation start (at time 0) in the 
in-control state and the out-of-control state simultaneously. Thus, the prejudice of assuming the 
observations are initially in the in-control state or the out-of-control state is removed. Further, 
Theorem [T] introduces the upper boundary h that prevents the CUSUM chart rising too high. 



Gandy and Lau (2012) explain how using an upper boundary in non-restarting CUSUM charts 
helps detect periods of out-of-control and in-control activity. Although the upper boundary h 
has been introduced in Theorem [TJ its value is not specified. This practical issue is discussed in 
detail in Section [3] where we explore how varying the upper boundary affects the signal process. 



3 Upper Boundary 

In this section we investigate how varying the upper boundary h affects the number of false and 
correct signals made by the charts. We begin by classifying different scenarios that can occur by 
changing h. An illustrative example demonstrates how these scenarios leads to different signals 
for the same data. 



3.1 Signal Gap and Overlay 

Setting h = kjj-\-ki in Theorem [l] means that R% and R^ signal whenever k^ is crossed. We shall 
refer to this as the single threshold case. However, this need not be the case as Theorem [T] just 
stipulates that h > max(ku, k^). When h > kjj + ki then it is possible to simultaneously signal 
in-control and out-of-control. We shall refer to this case as a signal overlay. When h < k\j + ki, 
then no signal can occur when either chart is between h — kjj and ki,. We shall refer to this 
as a signal gap. Thus both a gap and overlay are intervals where the charts cannot signal in- 
control or out-of-control with confidence. If we consider a simultaneous signal as no signal due 
to ambiguity, the cases of signal gap and overlay are the same. Thus, henceforth we shall only 
consider h < kjj + k^. 
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To see the disparity between different values of the upper boundary h consider the following 
example. Let Xt ~ iV(— 1/2,1) when in the in-control state and Xt ~ N(l/2, 1) when in the 
out-of-control state at time t. In this example we set the thresholds at kjj = ki = 5. In 
practice, however, it is typical to set these thresholds by pre-specifying the average run length. 
This can be achieved by running simulations of the CUSUM charts and conducting a numerical 
search. Alternatively, one could approximate the distribution of the CUSUM chart by a discrete 



Markov chain and obtain the average run length using the transition matrix (Brook and Evans 



1972). Using the Brook and Evans (1972) approach with 100 states, the average in-control and 
out-of-control run length with k\j = ki = 5 is approximately 930 time units. In this illustrative 
example we choose the out-of-control periods as time 16 to 35 and time 51 to 60. In Figure [2] we 
plot the two CUSUM charts R\ and based on the same random realization of X\ , X2 , . . . 
for h = 6, 8 and 10. 

Figure [2] illustrates some important points. First choosing a different value of h leads to a 
different signalling process for the same data. This is clearly presented by the signal indicators 
under Figure 2a 2b and 2c Second, once the two CUSUM charts have coupled into a single 
chart in the single threshold case (h = kjj + k£), definitive signals are given. More precisely, in 



Figure 2c, after time 13, the chart signals either in-control or out-of-control. This is not the case 



where a signal gap occurs. Comparing Figure 2a and 2b, we see that as the signal gap increases, 
fewer incorrect signals are made at the expense of fewer overall signals. This point is explored 
further in Section |3.2| Lastly, in this example, the two CUSUM charts eventually coalesce or 
couple into a single chart. The time until coupling depends, not only on Fq and F\, but also on 
the upper boundary h. Coupling occurs at value or h by construction of the CUSUM charts. 
Moreover, if we were the run CUSUM charts of the form ([!]) starting from every R$ £ [0, h] all 
charts will couple either at or h. This follows from the fact that R\ and R^ represent the 
"extreme" CUSUM charts. Thus at the time of coupling we signal confidently in-control, if at 
0, or out-of-control, if at h, since all possible charts of the form ([!]) agree. 



3.2 Practical Implications 

Although we have discussed the scenarios that arise with different choices of the upper boundary, 



we have yet to explore the practical implications. As illustrated in Section 3.1, the choice of h 
affects the signals made by the charts. We now explore how the choice of h affects the number 
of correct and false signals in a small simulation. 

We use the same in-control and out-of-control distributions and thresholds used Section 13.11 
The true in-control and out-of-control periods are represented by the grey areas in Figure [3j For 
a single iteration, we use the CUSUM charts Rf and R% with h = 6, 8, 10 all with the same 



seed. We repeated this 10,000 times. Figure 3a and 3b are plots the average number of false 



signals and correct signals, respectively, pointwise in time. 



In Figure 3a when switching between states, there is a sudden peak in the number of false 
signals which subsequently declines. The single threshold case where h = 10, leads to the an 
overall higher false and correct average than the other values of h. From Figure [3] the notion 
of a signal gap can be interpreted as follows: Whilst a chart is in a signal gap, signalling is 
deferred until a more "definitive" signal can be made i.e. when the chart exits the gap. This 
results in fewer false signals being made, with the trade-off that fewer signals overall are made. 
In practice, this could represent the users attitude to "err on the side of caution" and patiently 
waiting to signal with more confidence. 
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(c) h = 10 Single Threshold 

Figure 2: Upper figures contain one lower (solid) and upper (dashed) chart with lower (solid) 
and upper (dashed) threshold. Grey area represents the out-of-control period. Underneath is 
the signal indicator with grey areas representing signals. 



7 





1 10 20 30 40 50 60 70 80 90 

Time 

(b) Average Number of Correct Signals Pointwise in Time 



Figure 3: Average number of false and correct signals for h = 10 (solid): same threshold case, 
h = 8 (dotted) and h = 6 (dashed): both signal gap cases. 



4 Discussion 

We have shown that using non-restarting CUSUM charts of the form M are optimal in the 
sense of Q. We specifically use the two charts, R\ starting at and .Restarting at the upper 
boundary. This choice results in coupling of the CUSUM charts (see Figure [2J where after the 
charts coalesce, they remain exactly the same. It is clear that after coupling, only one CUSUM 
chart needs to be considered. Coupling would not occur when using two CUSUM charts that are 
restarted - this feature is specifically for non-restarting CUSUM charts with an upper boundary. 

A natural question to ask about coupling is: how long do the two non- restarting CUSUM 
charts take to couple? Denote the coupling time as T = min{t > 1 : R\ = Rf}. Denote 
i/f = min{i > 1 : R\ = h} and ui = min{t > 1 : R^ = 0} as the first time when the lower 
CUSUM chart reaches the upper boundary and the upper CUSUM chart reaches respectively. 
As T = min(z/ T , 1/4.) it follows that E(T) < mxn(Efa), E{y^)) for all PeP, since all CUSUM 
charts of the form ([T]) with any starting value, couple either at value or the upper boundary. 
For such that £>{Xt) = F\ for all t G N we would expect 

E(T) < E{u t ) < E( V]e ) 
and similarly for P £ V such that C{X t ) = F all t 6 N, 

E(T) < E( n ) « E(v t ). 
Proving such coupling results could be a topic for further research. 
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Appendix 1 

Proof of Theorem [T] 

We proceed to show the following: for any signal process Z and j G {0, 1} 

C j (Z) > T> j (Z) > T> j (Z j ) = C j (Z*). 

We start by showing C 1 (Z) > T) 1 (Z) for any signal process Z. As a first step we show that 

C\Z) > (5) 
sup max {esssup E ( [t\ (Z) - n + l] + | T n -i) : P G V, C(X t ) = Ftft > n } . 

Consider the stopping time in C^(Z), namely r^(Z) — n + 1 and \t\{Z) — n + l] + . For all j > 1 
and fix n G N we have 

{t^(Z) - n + 1 = j} = {Z m ^ 1 for n < m < {j + n - 1), Zj +n _i = 1} 

and 

{[^(Z) - n + 1]+ = j} = {Z m £ 1 for 1 < m < (j + n - 1), Z J+n _i = 1}. 

Thus { [rj 1 (Z) — n + 1] + = j} C {t^ (Z) — n + 1 = i} and so have a pointwise order of the stopping 
times. This implies ([5]). 
Next we show that 

sup max{ esssup E ( [t{ (Z) -n + l] + | P n -i) ■ P&V, C{X t ) =Ftft > n\ 
neN IV/ J 

> T>\Z). (6) 

For a fixed n G N 

max {esssup £ (Jr/ (Z) -n + l] + | J^-i) : PeV, C{X t ) = F x \/t > n} (7) 
> esssup e([tI(Z) -n + l] + \P n ^ = (Z) 

where, on the right hand-side of the inequality, is under P G V such that C(Xt) = Fq for all 
t < n and C(Xt) = F\ for all t > n. Taking the supremum of ([7]) over n G N shows ([6]). This 
completes showing C 1 (Z) > D 1 (Z). A similar argument can be used to show C (Z) > D°(Z). 

Since B 3 (Z) > c ? - it fo llows that B^(Z) > Cj for j G {0, 1}. As B-[{Z) is the expectation in 
Q it follows that flRitovj |l990[ Proposit ion 2) B j (Z) > B j (Z j ) for any signal process Z. 

A consequence of |Moustakides| fll986[ Lemma 1) is that T) l (Z l ) = D}(Z 1 ). Thus, by defini- 
tion of D^Z 1 ), 

D^Z 1 ) = E (inf{t > 1 : > k L } \ P G V : C{X t ) = Ftft G N) . (8) 
Replacing inf{i > 1 : > k^} in Q with inf{i > 1 : R\ > ki} as they are the same, gives 

D^Z 1 ) = C](Z*). 

Thus, to show D^Z 1 ) = C X {Z*), it suffices to show C l {Z*) = C*(Z*) for all neN. 



9 



To show this we follow an argument similar to that used in Moustakides (1986 ( Lemma 1). 
For any m > n > 1 and for fixed {X n+ i, . . . , X m }, the quantity is a non-decreasing function 
of R n . This implies that the stopping time t^(Z*) = inf{i > n : Rf > k^} is non-increasing 
with Rn-i- Thus the essential supremum in C*(Z*) is achieved for R n -\ = 0. Moreover, the 
essential supremum in (Z*) remains unchanged when taking the maximum over FgP such 
that C{Xt) = F\ for t > n. Hence from stationarity all C*(Z*) are equal. This completes 
showing D^Z 1 ) = C^Z*). 

We now show that D°(Z°) = D°(Z*). Similarly, we have D°(Z°) = D°(Z°) thus by defini- 
tion of D?(Z°) 

D°(Z°) = E(M{t > 1 : SY > ku} | P G V : = F V£ G N). (9) 

Replacing inf{i > 1 : > A;^/} in (|9]) with inf{£ > 1 : ii^ < h — ku} as they are the same, 
gives D°(Z°) = C?(Z*). Showing that C°(Z*) = C°(Z*) for all n G N follows from the same 
argument given above. Thus D°(Z°) = C°(Z*). 
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